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Abstract 

Study abroad is a growing phenomenon in higher education. Although such growth is typically lauded, efforts to 
measure the impact of international experiences on student learning have been limited. This study assesses the 
academic benefit of a study abroad program, offered by a U.S. university, with measures of self-reported learning. 
Study abroad participants were tested before and after completing their program and results are compared to a 
control group that did not study abroad. Study abroad students reported significantly greater knowledge than 
stay-at-home students on two of five cognitive dimensions. Further, study abroad students reported greater 
knowledge upon return than they did pre-departure on all five cognitive dimensions. The enhancements in 
knowledge were not significantly affected by sex, major, or ethnicity. Results of the study offer support for the 
value of study abroad in improving student perceptions of the learning experience. Implications for researchers 
and practitioners are discussed including additional steps that could be taken for study abroad assessment. 
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1. Introduction 

Participation in study abroad is growing at a rapid pace.The 2011 Open Doors Report shows that overseas study 
by U.S. students was up in 2010 (the most recent available year) by over 10,000 students compared to the 
previous year (270,604 versus 260,327) (HE, 2011). U.S. students studying abroad have more than tripled over 
the past two decades. The Open Doors report also emphasized that study abroad is now taking place in a wider 
range of locations and representing a wider range of academic disciplines than in the past. In addition to the 
rapid growth, there has been an explosion in the types of study abroad programs available to students. We can 
distinguish these programs by length of sojourn, language and cultural preparation, primary motivations of 
learners, degree of socio-cultural difference, second language use, level of learning structure/support; type of 
residence, degree of cultural contact (immersion), degree of self-direction, and degree of cultural analysis 
(Slimbach, 2010). Our large public university in the western U.S. has mirrored these trends. We have 
experienced substantial and sustained growth in our study abroad program. Study abroad participation has 
increased nearly 1000 percent over the past twelve years. Even more importantly, the perception of study abroad 
has changed. It is no longer seen as just an optional add-on experience. The university currently has 37 majors 
and academic programs that include an international experience as a graduation requirement. These majors are 
diverse, from Nursing to Liberal Studies (pre-teacher education) to International Business, and reflect the 
academic value faculty and administrators place on global learning experiences. Study abroad is widely and 
highly regarded beyond our university and has been identified as a "high impact educational practice" by those 
who study student success rates in the National Survey of Student Engagement (NSSE), a practice that can lead 
to increased rates of student retention and student engagement (Kuh, 2008). 

There are many potential approaches for assessing the value or impact of the study abroad experience. It may be 
useful to investigate, for example, how the existence of a robust study abroad program impacts the institution as 
a whole. Does on-campus instruction become more internationalized because many faculty and students are 
engaged in study abroad? It is also important to measure the impacts of specific study abroad programs and 
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specific program attributes (for e.g., homestay vs. apartment living, program length and type) and whether those 
are successful in a given regional and cultural context. On the individual student level, it is important to look at 
the degree of satisfaction with the study abroad experience as well as retention and graduation rates. Although 
valuable, however, these studies do not directly measure academic benefit (Sutton & Rubin 2004). Research 
efforts should also be directed toward understanding student learning as a result of study abroad, heeding 
Steinberg’s counsel that “international educators need to demonstrate that the students on our programs return 
home having grown intellectually and personally, having developed a greater measure of global and intercultural 
competence and, when relevant, having developed greater fluency in languages other than English. We also need 
to demonstrate that they are much better prepared for careers in a global environment” (Steinberg, 2007, p. 19). 

Anecdotal evidence of student learning through the study abroad experience is rich. Students returning from 
study abroad frequently describe their experiences as “life-changing” and “transformative.” Students often report 
improved academic and cognitive growth, along with intercultural competencies and psycho-social development. 
However, study abroad practitioners are beginning to see a need to more accurately assess the impact of study 
abroad as an academic endeavor. We desire a better understanding of what our students learn through study 
abroad experiences and to be able to document the value of study abroad in more than just anecdotal fashion. 
Just as academic programs and courses on campus are assessed for student learning outcomes and effectiveness, 
student learning through study abroad should also be assessed. Anecdotal evidence needs to be improved upon 
by providing empirical evidence if we hope to increase our understanding of the impact of study abroad. 

There are several reasons why empirical evidence about students’ learning throughout study abroad is useful. 
First, it allows institutions to measure the success of internationalization efforts by evidence of learning rather 
than by an increase in the number of study abroad participants. Although participation rates are useful, they do 
not, in and of themselves, suggest a meaningful outcome of internationalization and international education 
(Deardorff, 2005). Study abroad is a relatively new field of research (Bolen, 2007, p. 2) and most institutions 
offer a wide range of programs. Hence, it is important to know which of the many types of study abroad 
programs that institutions offer advance their mission as it is related to student learning outcomes such as 
intercultural competency, global citizenship, and transformational learning. Such evidence could provide 
information to those in charge of designing and implementing study abroad programs about the features or 
characteristics that best promote student learning. This, in turn, may allow practitioners to improve learning in 
current study abroad programs, perhaps by adding new pre-departure and re-entry components to supplement 
learning opportunities, and provide a road map for intentionality in designing new study abroad programs. 
Another reason that empirical evidence is useful is in responding to critics of international education who feel 
that resources are better directed toward on-campus programs. These critics may be skeptical about the value 
study abroad adds to a college education. Empirical evidence of increased student learning reduces the likelihood 
that study abroad will be seen as "party abroad" and substantiate the value and depth of these programs. 

The purpose of this study is to investigate students’ perceptions of what they learn as a result of the study abroad 
experience. We focus specifically on cognitive learning outcomes in order to better understand the academic 
impact of study abroad. 

2. Literature Review 

Historically, most of the research questions in the literature on study abroad have been related to the teaching 
and learning of foreign languages and the cultural learning that coincides with those activities (Brandt & Manley, 
2002). It has been uncommon for assessment to focus on cognitive, academic outcomes of international 
education (Rubin & Sutton, 2004). Even less common have been studies that assess students before and after 
their study abroad experience. Still rarer are studies that use control groups (Stronkhorst, 2005). 

2.1 Before and After Studies 

Recently an increasing number of studies have emerged that assess students before and after their study abroad 
experiences. Emert and Pearson (2007) indicate reported improvement in intercultural competency growth in 
students who studied abroad and participated actively. In Nagy’s (2008) research, participants reported becoming 
independent and improving their English proficiency. Rexeisen and Al-Khatib (2009) found that study abroad 
has a positive impact in students’ development of cross cultural sensitivity and environmental attitudes but did 
not have a direct impact on moral reasoning. 

2.2 Control Group Studies 

Among the minority of studies that have used control groups in assessing study abroad learning outcomes are the 
ones conducted by Clarke, III, et ah, (2009); Kehl & Morris (2007) and Sutton & Rubin (2004). Sutton & 
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Rubin’s (2004) work is the basis of the current research. Both Clarke et al. (2009), and Kehl & Morris (2007) 
administered the Global-Mindedness Scale (Hett, 1993) to a sample and a control group. Clarke et al, also 
administered Olson & Kroeger’s (2001) Intercultural Sensitivity Index (ISI) to both samples. The conclusion of 
both studies was that students who studied abroad became more globally minded than students who stayed at 
home. Clarke et al., also found that students who study abroad had greater intercultural proficiency and increased 
openness to cultural diversity in comparison to those students who stayed at home. 

The Sutton and Rubin (2004) GLOSSARI study documented self-reported cognitive, academic outcomes of 
international education between those who participated in study abroad and non-participants among students 
attending sixteen different public institutions within the Georgia State University System. 

2.3 Before and After with Control Group 

Some studies in the study abroad literature include both pre-post assessment and a control group. In this category 
is research conducted by Stronkhorst (2005), Williams (2005), Elola & Oskoz (2008), and Vande Berg (2009). 

The focus of the Stronkhorst (2005) study was to assess language competencies (English) and multicultural 
personality (MP) characteristics, (i.e., cultural empathy, open-mindedness, social initiative and intercultural 
competencies) in two groups of students from two Dutch institutions. Eighty two students participated in the 
study. The effects of the international mobility among the students who participated in Stronkhorst’s research 
were mixed. Many internship students made considerable and/or reasonable progress in both language 
proficiency and MP characteristics while study abroad students' progress was limited. For quite a few of these 
students their international experience had a negative impact on their MP characteristics. The majority of the 
students who completed an internship abroad stated that they had achieved limited or no progress in the area of 
professional competency. Students in the control group did not rate their progress higher. 

Williams administered the Cross-Cultural Adaptability Inventory (CCAI) and the Intercultural Sensitivity 
Inventory (ISI) both pre and post-sojourn to 52 students, 27 of whom had studied abroad and a convenience 
sample of 25 students who stayed at their home campus. The study revealed that students who studied abroad 
displayed a greater change in intercultural communication skills after their semester abroad than students who 
stayed on campus. The result of the research also indicated that the greatest predictor of intercultural 
communication skills was exposure to other cultures. 

The purpose of the Elola & Oskoz (2008) study was to examine how intercultural competency develops among 
study abroad and non-study abroad students. Results found that blog interactions had a positive effect on both 
group’s intercultural competence with no significant differences between the two groups. 

Vande Berg (2009) conducted a four-year study to find out if U.S. students who study abroad perform better 
when educators proactively intervene in their learning or if they learn effectively if they are left to their own 
devices. The study found that students who studied abroad improved their language learning at a level 
significantly higher than students in the control group. Regarding cultural development, the study abroad 
participants, on average, had significantly greater gains than the control group participants. Finally, the study 
found that intervening in students' learning with elements such as a pre-departure orientation with a cultural 
component, provided “sufficient conditions” to increase learning over students who stayed at home. 

In summary, in recent years an increasing number of studies have emerged that assess students either before and 
after their study abroad experience or use control groups. It is still uncommon to find studies that do both. Such 
studies are important for assessing the value added that study abroad opportunities provide students. The pre-test 
vs. post-test comparison tests whether some value manifested in the duration of the program is significant. The 
presence of a control group helps to ensure that improvement amongst study abroad students is attributable to the 
experience above and beyond similar peers who choose not to participate in study abroad. 

We seek to implement both of these desirable methodological interventions in the present study as we assess 
study abroad learning outcomes through an empirical replication of portions of the GLOSSARI project. The 
ultimate goal of this study is to “confront the challenges of a data-driven, evidentiary-based articulation of the 
values gained from study abroad” (Sutton & Rubin, 2004, p. 76). 

3. Hypotheses and Measures 

The primary focus of this study is to identify what students learn as a result of study abroad experiences, and 
how that compares with students who do not study abroad. Based on the literature reported, it is hypothesized 
that: 
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Hypothesis 1: Students who study abroad will report a significantly higher level of knowledge upon return than 
students who do not study abroad. 

Hypothesis 2: Students who study abroad will report a significantly higher level of knowledge upon return than 
they report prior to departure. 

Further, the study asks the question, "Is student learning as a result of study abroad impacted by demographic 
characteristics?". 

The present research was supported by a grant from the University System of Georgia to undertake replication of 
two phases of their Georgia Learning Outcomes of Students Studying Abroad Research Initiative (GLOSSARI) 
led by Sutton and Rubin. Sutton and Rubin published an early version of their model, which identified seven 
factors of student learning/knowledge (2004), but a revised and unpublished factor analysis (New Standard 
Version of the ILO, received via email from author Donald Rubin) yielded five meaningful dimensions. In 
particular, improvement is expected in the following areas: 

1) Functional knowledge: the skills and knowledge based needed to effectively function and navigate daily 
routines in a new environment. 

2) Knowledge of world geography: practical knowledge of physical and political geographic features. 

3) Knowledge of global interdependence: knowledge of the impact that actions in one part of the world have on 
another part. 

4) Knowledge of interpersonal accommodation: understanding of the need for flexibility and patience in 
communicating with other cultures. 

5) Knowledge of cultural sensitivity: understanding of how language and cultural contexts impact the ability to 
relate to and communicate with individuals of other cultures. 

This study will also consider other factors that may have an impact on student learning across the areas listed 
above. Three other possible predictors of change have been identified (sex, ethnicity and major) so that each can 
be tested to determine what change, if any, is caused by these factors. 

4. Procedure 

This study measures change via self-reported assessments of knowledge, skills, and abilities both before going 
abroad and in a post-test of those same knowledge, skills, and abilities once participants return. The researchers 
worked with the university’s offices of Enrollment Services and Institutional Research to run demographic 
statistics on the pool of study abroad students. From this data, they designed a representative control group that 
would remain on campus during the Fall semester (controlling for ethnicity, sex, student level and academic 
college) and administered the same pre-test and post-test at either end of the on-campus semester. 

The instmment employed is the GLOSSARI ILO survey, made available by the University System of Georgia. 
The GLOSSARI ILO survey is comprised of demographic questions and 29 statements. Participants were asked 
to select how strongly they agree or disagree with each statement, on a 5-point Likert scale. The GLOSSARI 
ILO instalment is designed to measure the 5 dimensions of student learning identified by Sutton and Rubin. For 
each dimension, 2 to 14 statements are included. Sample statements from each dimension are: 

• Functional knowledge: I know how to locate basic information in a foreign newspaper. 

• Knowledge of world geography: I know the capital cities of at least 6 countries in South America. 

• Knowledge of global interdependence: 1 understand how foreign manufacturing affects the prices of consumer 
goods (e.g., clothing) in the U.S. 

• Knowledge of interpersonal accommodation: I know how to be patient when interacting with people. 

• Knowledge of cultural sensitivity: I am sensitive to my own reactions to people from different language and 
cultural backgrounds. 

5. Sample Population and Data Collection 

Two groups of students were studied including a study abroad group and a control group. The study abroad 
group consisted of students participating in short-term and semester-long study abroad programs during 
consecutive summer and fall semesters in 2009. Study abroad participants represented several program models, 
including short-term faculty-led programs, semester exchanges, and non-university programs. The control group 
was made up of students that did not study abroad during the same summer and fall semesters and was selected 
by the university’s Enrollment Services office to be representative of the study abroad group in terms of sex, 
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ethnicity, student level, and academic discipline (by college). Table 1 shows the demographics of the invited 
sample. 

Students in both groups were invited by email to participate in the survey, which was administered electronically 
through a survey management site. To increase the reliability of the data, students were asked to complete the 
survey within a four week window. For example, students in the study abroad group were given access to the 
survey two weeks prior to their departure and access was withdrawn two weeks after arrival in their assigned 
country. A similar method was employed for the control group remaining on campus. Survey participation was 
completely voluntary, but small incentives ($20 bookstore gift cards) were provided. Each participant was 
assigned a random tracking number, which allowed the investigators to match their pre-test and post-test 
responses while keeping identifying information separate from actual responses. Students that completed the 
pre-test received an invitation to complete the post-test within a two-week window of their return to campus or 
the end of the campus semester. Of the 183 students in the study abroad group, 81 students participated by 
completing the pre-test of which 73 of the responses were complete and usable. Fifty eight of the study abroad 
pre-test students followed through with the post-test including 53 responses that were complete and usable. Of 
the 300 students in the control group, 39 students participated by completing the pre-test of which 33 of the 
responses were complete and usable. Twenty one of the control group pre-test students followed through with the 
post-test and all 21 responses were usable. Thus, the study abroad group had a pre-test usable response rate of 
39.9 percent (73/183) and a post-test usable response rate of 29.0 percent (53/183) whereas the control group had 
a pre-test usable response rate of 11.0 percent (33/300) and a post-test usable response rate of 7.0 percent 
(21/300). The response rate among the study abroad group was satisfactory but the rate was lower than desired 
among the control group. 

6. Results 

The study abroad and control group respondents were initially compared with respect to their pre-departure 
perceived levels on the five learning outcome dimensions identified in the refined factor analyses by Sutton and 
Rubin. This analysis parallels Phase I of the original GLOSSARI study (Sutton and Rubin, 2004). Summary 
statistics for the 5 dimensions are presented Table 2. Two tests were completed to compare study abroad with 
control group students on each dimension. The first was an F-test for equality of variances and the second was a 
t-test for equality of means. The t-test assumed unequal variances in the event of a significant F-test. The result 
of these tests appears in Table 2 and there were no significant differences between the two groups in terms of 
their mean responses along any of the five dimensions. This non-finding suggests our steps for identifying a 
suitable control group for the study abroad students were successful. That is, because the present study obtained 
semester start/pre-departure as well as semester end/upon return measures it was possible to substantiate that the 
two groups were similar in their initial levels on the five dimensions. There was a significant difference in the 
variances along the functional knowledge dimension, however, for which we have no suitable explanation. 


Table 1. Sample demographics of students invited to participate in the study 




Study Abroad (n=183) 

Control (n=300) 


Gender 

Male 

117 

63.9% 

192 

64.0% 


Female 

66 

36.1% 

108 

36.0% 

Student Level 

Freshmen 

5 

2.7% 

8 

2.7% 


Sophomore 

42 

23.0% 

69 

23.0% 


Junior 

78 

42.6% 

128 

42.7% 


Senior 

58 

31.7% 

95 

31.7% 

College 

Flealth Services 

4 

2.2% 

7 

2.3% 


Business 

56 

30.6% 

93 

31.0% 


Education 

2 

1.1% 

4 

1.3% 


English 

3 

1.6% 

6 

2.0% 


Arts & Letters 

48 

26.2% 

75 

25.0% 


Science 

15 

8.2% 

25 

8.3% 
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UG Division 


3 


1.6% 

5 

1.7% 

Prof. & Fine Arts 


52 


28.4% 

85 

28.3% 

Ethnicity American Indian 


1 


0.5% 

2 

0.7% 

African American 


8 


4.4% 

6 

2.0% 

Mexican American 

24 


13.1% 

51 

17.0% 

Other Hispanic 


7 


3.8% 

11 

3.7% 

Asian 


6 


3.3% 

16 

5.3% 

SE Asian 


1 


0.5% 

7 

2.3% 

Pacific Islander 


2 


1.1% 

3 

1.0% 

Filipino 


4 


2.2% 

20 

6.7% 

White 


102 

55.7% 

148 

49.3% 

Other/NS 


26 


14.2% 

25 

8.3% 

International 


2 


1.1% 

11 

3.7% 

Table 2. Summary statistics and 

comparisons of 

variances and means between 

study abroad and 

non-study-abroad students prior to departure (at the beginning of the semester) 


Study 

Variable (n=75) 

M 

Abroad 

S 

Non-Study Abroad 
(n=33) 

M S 

F-test for 

equality of o 2 

t test for equality 
of p 

Functional Knowledge 46.59 

8.40 


43.15 

12.20 

F(74,32)=0.47** 

t(45)=l .47 

World Geography 17.31 

4.41 


16.73 

4.34 

F(74,32)=1.03 

t(106)=0.64 

Global 17 55 

Interdependence 

3.49 


17.70 

3.01 

F(74,32)=1.35 

t(106)=-0.23 

Interpersonal ^ 

Accommodation 

1.11 


8.45 

1.25 

F(74,32)=0.79 

t(106)=0.68 

Cultural Sensitivity 11.17 

Note: **p<.01 

2.27 


11.41 

1.98 

F(74,32)=1.32 

t(106)=-0.53 

Table 3. Summarystatistics and 

comparisons of variances and 

1 means between 

study abroad and 

non-study-abroad students after return (at the end of the semester) 



Study 

Variable (n=53) 

M 

Abroad 

S 

Non-Study Abroad 

(n=21) 

M S 

F-test for equality 
ofo 2 

t test for equality 
of p 

Functional 

Knowledge 

6.93 


46.52 

13.39 

F(52,20)=0.27*** 

t(24)=3.69*** 

World Geography 18.06 

4.25 


16.38 

4.49 

F(52,20)=0.90 

t(72)=l .47 

Global 18 49 

Interdependence 

3.26 


17.81 

3.12 

F(52,20)=1.09 

t(72)=0.84 

Interpersonal ^ 

Accommodation 

1.16 


8.95 

.97 

F(52,20)=1.41 

t(72)=0.46 

Cultural Sensitivity 12.74 

1.78 


11.48 

2.16 

F(52,20)=0.68 

t(72)=2.37* 


Note: *p<.05, ***p<.001 


Hypothesis 2 was tested by comparing the before departure responses against the return responses for the study 
abroad students. This hypothesis examines whether these students improved along these dimensions during their 
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study abroad experience. Paired sample t-tests were used to compare the mean responses of the study abroad 
students along these five dimensions. Hypothesis 2 was supported on all 5 dimensions. That is, study abroad 
students reported statistically significant higher mean responses along the five dimensions upon returning home 
relative to departure. 


Table 4. Paired sample t tests for students before and after study abroad 


N=53 

Before Study Abroad 

M S 

After Study Abroad 

M S 

t test for equality of |! 

Functional Knowledge 

45.85 

8.44 

57.85 

6.93 

t(52)= 

-12.00*** 

World Geography 

17.34 

4.73 

18.06 

4.25 

t(52)= 

-1.80* 

Global Interdependence 

17.32 

3.50 

18.49 

3.26 

t(52)= 

-2.90** 

Interpersonal 

Accommodation 

8.58 

1.10 

9.08 

1.16 

t( 52 )— 

-2.30* 

Cultural Sensitivity 

11.38 

2.02 

12.74 

1.78 

t(52)= 

-4.04*** 

Note: *p<.05, **p<01, * 

**p<.001 







Although not specifically hypothesized in the current study, we also conducted paired sample t-tests for control 
students over the same period of time (one semester) in order to emphasize that increases in knowledge for the 
Study Abroad students were likely attributable to their international experience rather than simply the passage of 
time. In terms of raw means, the control students reported lower means on two dimensions (world geography and 
cultural sensitivity), higher means on two dimensions (functional knowledge and interpersonal accommodation) 
and no difference on one dimension (global interdependence). There was a statistically significant improvement 
along the dimension of interpersonal accommodation. 

To address the research question regarding demographics the data was also analyzed to determine whether there 
were any significant effects from respondent sex, major, or ethnicity. No such effects were found to exist at 
either the semester start/pre-departure point nor at the semester end/upon return point hence offering support that 
differences were the result of the study abroad experience and not demographic differences. 

7. Discussion 

The results offer varying degrees of support for the hypotheses. With regard to Hypothesis 1, which compared 
students who studied abroad to those who did not, support was observed for the perceived value of an 
international experience in learning to navigate daily routines in a new environment, i.e., functional knowledge. 
Although the two groups of respondents indicated no significant difference in their perceived levels of ability on 
this dimension prior to the departure of the study abroad group, upon return the study abroad students differed 
significantly from those who had stayed home with respect to their self-evaluations on this dimension. They 
believed they had moved forward on this dimension while the control group did not. This finding is consistent 
with that of Sutton & Rubin in the initial GLOSSARI study (2004) who also noted a significant difference 
between those who studied abroad and those who did not on the dimension of functional knowledge. Perceptions 
of knowledge regarding cultural sensitivity were also found to be significantly enhanced for students who 
studied abroad. This finding suggests that the experience of studying and living in a foreign environment not 
only builds confidence in navigating basic living skills but also increases individuals’ beliefs in their abilities to 
be introspective with respect to their reactions and personal styles in culturally diverse settings. One might argue 
that this outcome will be potentially valuable as the global economy continues to build and the likelihood of 
today’s students competing in that economy increases as well. 

With regard to Hypothesis 2, which compared students who studied abroad before and after their experiences, 
statistically significant increases were observed across all five dimensions. Students who studied abroad 
demonstrated significant increases in the perceived enhancement of their skills on all of the five dimensions 
studied while those who did not study abroad perceived significant differences in their knowledge on only one of 
the five learning outcomes. Upon return, study abroad students perceived that their functional knowledge was 
stronger than they had believed it to be before they left - they returned believing they were better able to 
accomplish basic tasks such as locating a restaurant, taking public transport, and purchasing small items in a 
foreign country as well as more advanced skills such as comparing languages and cultures. Study abroad 
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students also indicated significant differences in their knowledge regarding interpersonal accommodation 
suggesting that strength was believed to have been gained in areas such as patience and flexibility. Further, 
studying abroad was positively associated with increases in beliefs that cultural sensitivity was enhanced. Study 
abroad students felt they returned with a significantly increased understanding of their reactions in different 
settings and the significance of language and cultural differences. Perceptions of knowledge regarding the 
interdependence of countries around the globe also increased significantly. Perceptions of world geography were 
also increased albeit this was a rather weak but still nonetheless a statistically significant finding. In sum, 
Hypothesis 2 was fully supported by the data. 

The design of the study led to a number of statistical considerations, two of which are particularly important for 
interpreting the results. First, because 10 separate hypothesis tests were conducted (2 research hypotheses x 5 
dependent variables each), the experiment-wise type I error rate is high. A conservative approach for adjusting 
for this is to divide a by the number of tests which would, in this case, result in a = .005 (i.e., .05 10). Using 

this more restrictive standard the cultural sensitivity difference reported in Table 3 for Hypothesis 1 would no 
longer be statistically significant, leaving only the dimension of functional knowledge. Further, the differences 
yielded for the dimensions of world geography and interpersonal accommodation, as reported in Table 4, would 
no longer be statistically significant for Hypothesis 2, leaving functional knowledge, global interdependence, and 
cultural sensitivity. Thus, we note that the findings are more robust for certain dimensions with respect to the use 
of an error-rate adjustment. Nonetheless, both hypotheses receive some degree of support with the more 
restrictive approach. 

Another consideration of the design was the effort made to use anonymous tracking numbers which allowed for 
pairwise comparisons to be made for Hypothesis 2. The paired sample t-test is known as an especially robust 
statistical test and is less prone to type II errors than is the independent sample t-test used in Hypothesis 1. This 
is demonstrated, for example, by the findings reported for the global interdependence dimension. The difference 
between the study abroad group and control group is non-significant in Table 3, suggesting no improvement. The 
paired sample t-test, however, reveals a robust statistically significant difference within the study abroad students 
(and not within the control group students) as reported in Table 4. Thus, this measurable improvement would 
likely have gone undetected without the before and after repeated comparisons and hence demonstrates the value 
of tracking individual students as they progress through a program. 

8. Conclusions 

The current study’s findings are of considerable importance to assessment efforts on campus and may also have 
implications for other higher education institutions facing similar challenges. First, the study represents a 
bona-fide programmatic assessment effort at the individual student level of analysis, a condition which is being 
increasingly demanded from our state, campus-wide administration, and from many of our colleges’ professional 
accreditation associations. Prior to this study, success was gauged primarily through growth in study abroad 
participation rates as well as anecdotal reports of satisfaction from the participants, neither of which fully 
substantiated the value-added gains demonstrated in this study. Second, the study raises some concern for the 
educational experience of the students not participating in study abroad. As educators, we actively seek to teach 
international skills and knowledge to all of our students and although study abroad is a highly effective means to 
achieve this, we are nonetheless motivated to seek methods for instilling these learning outcomes in our students 
who do not study abroad. The findings reported here simultaneously validate the value of our study abroad 
program and raise concerns about the lack of international experience among our domestic bound students. 

In addition, this study further builds the existing body of knowledge that is attempting to provide evidence of the 
value of study abroad through the measurement of its effect on specific learning outcomes. Today’s international 
educators have moved beyond general feelings that “study abroad is a good thing to do” and toward 
documenting significant learning outcomes that can be expected from such experiences. This approach, if 
successful, can be expected to enhance the push from institutions of higher education for their students to engage 
in study abroad, which has already been identified as one of several “deep learning” activities by the National 
Survey of Student Engagement Institute (Kuh, 2008). It can also be expected to encourage intentionality in 
program design, as international educators develop an evidence-based understanding of what types of study 
abroad programs and program elements best contribute to student learning. 

Evidence of learning is only partially determined by students' self-reports, however. The next step in exploring 
and potentially affirming the value of study abroad on specific learning outcomes is to move from perceptual 
measures to direct measures of learning. The effort to support the value of study abroad through evidence of its 
efficacy in enhancing student learning is stronger with direct and perceptual measures than with perceptual 
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measures alone. The value of students reporting that their knowledge of world geography or global 
interdependence, for example, is enhanced as a result of studying abroad is not as strong as direct evidence that 
such knowledge has indeed increased. Studies that utilize direct measures of student learning often take 
considerable time to execute and typically draw on a more limited sample of study abroad participants; 
nevertheless, they make a valuable contribution to our understanding of student learning. Future studies may 
incorporate a direct measure component in order to better understand the relationship between perceived and 
actual learning abroad. 

By analyzing students who have studied abroad and those who have not both pre and post, the current study 
hopes to provide support to counter an often heard argument that students who study abroad are simply different 
to begin with - more aware, more privileged, more academically elite. Given that no statistically significant 
differences were found between the study abroad group and the control group at the beginning of the study, it is 
not unreasonable to assume that any differences in perceived improvement of learning at the end of the study 
came as a result of the study abroad experience. Students that studied abroad reported significantly higher rates 
of perceived improvement across all learning outcome measures after going abroad. The findings reported here 
add to the anecdotal evidence which suggests that study abroad is a worthwhile educational endeavor that has 
significant self-perceived impacts on student learning. 
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